Performance Of Machine Learning Methods Research Articles

Investigating the potential of Machine Learning for fault location on Inverter-Based Resource interconnection lines: Insights and recommendations

Given the increasing penetration of Inverter-Based Resources (IBR) and the impacts of these generators on existing fault location functions, this paper explores the potential of Machine Learning (ML) for designing one-terminal fault locators applied to systems with IBRs. For the studies, a system with a widely used topology for IBR interconnection to a transmission grid is modeled in PSCAD software, considering fault scenarios on the interconnection line with varying fault parameters, in addition to different IBR controls/topologies, grid short-circuit levels and signal noise levels. Besides the investigations on correctly choosing the attributes extracted from the signals for training ML methods, which introduces the concept of learning based on the existing method’s response, the studies cover performance assessments of 35 ML methods for the fault location task, indicating the most suitable ones, and studies about the influence of the training set size on the ML methods performance. Moreover, comparative performance analyses are conducted between the ML-based methodology and state-of-the-art fault locators, highlighting the benefits of the ML-based approach in reducing errors obtained in the fault location task. Finally, the paper recommends stages for applying ML-based methods to real systems, besides providing insights and recommendations on such practical applications.

Multi-source precipitation estimation using machine learning: Clarification and benchmarking

Multi-source precipitation estimation techniques, including interpolation, bias correction, and merging, are essential for acquiring high-precision precipitation datasets. Machine learning (ML) methods have achieved inspiring success in this field, but current studies utilizing ML often blur the boundaries of interpolation, bias correction, and merging because of the utilization of uniform input–output frameworks. The ambiguity in classification (interpolation, bias correction and merging) results in inconsistent and inappropriate comparative benchmarking among studies. A widespread example is the tendency to select easy-to-beat benchmarks, which can magnify the relative performance of ML methods. We propose to reclarify ML-based precipitation estimation by conducting a series of experiments with designed feature combinations to unveil the essence of ML-based methods. Our findings suggest that with different feature combinations, there are no essential distinctions between models for merging, bias correction and interpolation. Ground measurements, as the reference in supervised learning, play a more vital role than different features in precipitation estimation. This means that the ML-based multi-source merging or bias correction should be treated as gauge data-based interpolation with various auxiliary predictors. Moreover, the improvements of ML estimates are more significant compared to low-standard benchmarks like individual precipitation datasets and simple merging methods, but less preeminent compared to high-standard benchmarks such as a sophisticated interpolation method (e.g., Geographically Weighted Regression). Therefore, we call for positioning such ML applications as interpolation and adhering to high-standard benchmarks whenever possible, which can illustrate the true potential and promote further advancements of ML in geospatial estimation.

Machine learning model and nomogram to predict the risk of heart failure hospitalization in peritoneal dialysis patients

Introduction The study presented here aimed to establish a predictive model for heart failure (HF) and all-cause mortality in peritoneal dialysis (PD) patients with machine learning (ML) algorithm. Methods We retrospectively included 1006 patients who initiated PD from 2010 to 2016. XGBoost, random forest (RF), and AdaBoost were used to train models for assessing risk for 1-year and 5-year HF hospitalization and mortality. The performance was validated using fivefold cross-validation. The optimal ML algorithm was used to construct the models to predictive the risk of the HF and all-cause mortality. The prediction performance of ML methods and Cox regression was compared. Results Over a median follow-up of 49 months. Two hundred and ninety-eight patients developed HF required hospitalization; 199 patients died during the follow-up. The RF model (AUC = 0.853) was the best performing model for predicting HF, and the XGBoost model (AUC = 0.871) was the best model for predicting mortality. Baseline moderate or severe renal disease, systolic blood pressure (SBP), body mass index (BMI), age, Charlson Comorbidity Index (CCI) score were strongly associated with HF hospitalization, whereas age, CCI score, creatinine, age, high-density lipoprotein cholesterol (HDL-C), total cholesterol, baseline estimated glomerular filtration rate (eGFR) were the most significant predictors of mortality. For all the above endpoints, the ML models demonstrated better discrimination than Cox regression. Conclusions We developed and validated a novel method to predict the risk factors of HF and all-cause mortality that integrates readily available clinical, laboratory, and electrocardiographic variables to predict the risk of HF among PD patients.

Machine-learning algorithms in screening for type 2 diabetes mellitus: Data from Fasa Adults Cohort Study.

The application of machine learning (ML) is increasingly growing in biomedical sciences. This study aimed to evaluate factors associated with type 2 diabetes mellitus (T2DM) and compare the performance of ML methods in identifying individuals with the disease in an Iranian setting. Using the baseline data from Fasa Adult Cohort Study (FACS) and in a sex-stratified manner, we studied factors associated with T2DM by applying seven different ML methods including Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbours (KNN), Gradient Boosting Machine (GBM), Extreme Gradient Boosting (XGB) and Bagging classifier (BAG). We further compared the performance of these methods; for each algorithm, accuracy, precision, sensitivity, specificity, F1 score, and Area Under Curve (AUC) were calculated. 10,112 participants were recruited between 2014 and 2016, of whom 1246 had T2DM at baseline. 4566 (45%) participants were males, aged between 35 and 70 years. For males, age, sugar consumption, and history of hospitalization were the most weighted variables regarding their importance in screening for T2DM using the GBM model, respectively; these variables were sugar consumption, urine blood, and age for females. GBM outperformed other models for both males and females with AUC of 0.75 (0.69-0.82) and 0.76 (0.71-0.80), and F1 score of 0.33 (0.27-0.39) and 0.42 (0.38-0.46), respectively. GBM also showed a sensitivity of 0.24 (0.19-0.29) and a specificity of 0.98 (0.96-1.0) in males and a sensitivity of 0.38 (0.34-0.42) and specificity of 0.92 (0.89-0.95) in females. Notably, close performance characteristics were detected among other ML models. GBM model might achieve better performance in screening for T2DM in a south Iranian population.

Development and validation of survival prognostic models for head and neck cancer patients using machine learning and dosiomics and CT radiomics features: a multicentric study

BackgroundThis study aimed to investigate the value of clinical, radiomic features extracted from gross tumor volumes (GTVs) delineated on CT images, dose distributions (Dosiomics), and fusion of CT and dose distributions to predict outcomes in head and neck cancer (HNC) patients.MethodsA cohort of 240 HNC patients from five different centers was obtained from The Cancer Imaging Archive. Seven strategies, including four non-fusion (Clinical, CT, Dose, DualCT-Dose), and three fusion algorithms (latent low-rank representation referred (LLRR),Wavelet, weighted least square (WLS)) were applied. The fusion algorithms were used to fuse the pre-treatment CT images and 3-dimensional dose maps. Overall, 215 radiomics and Dosiomics features were extracted from the GTVs, alongside with seven clinical features incorporated. Five feature selection (FS) methods in combination with six machine learning (ML) models were implemented. The performance of the models was quantified using the concordance index (CI) in one-center-leave-out 5-fold cross-validation for overall survival (OS) prediction considering the time-to-event.ResultsThe mean CI and Kaplan-Meier curves were used for further comparisons. The CoxBoost ML model using the Minimal Depth (MD) FS method and the glmnet model using the Variable hunting (VH) FS method showed the best performance with CI = 0.73 ± 0.15 for features extracted from LLRR fused images. In addition, both glmnet-Cindex and Coxph-Cindex classifiers achieved a CI of 0.72 ± 0.14 by employing the dose images (+ incorporated clinical features) only.ConclusionOur results demonstrated that clinical features, Dosiomics and fusion of dose and CT images by specific ML-FS models could predict the overall survival of HNC patients with acceptable accuracy. Besides, the performance of ML methods among the three different strategies was almost comparable.

Surface-Enhanced Raman Spectroscopy-Based Detection of Micro-RNA Biomarkers for Biomedical Diagnosis Using a Comparative Study of Interpretable Machine Learning Algorithms.

Surface-enhanced Raman spectroscopy (SERS) has wide diagnostic applications due to narrow spectral features that allow multiplex analysis. We have previously developed a multiplexed, SERS-based nanosensor for micro-RNA (miRNA) detection called the inverse molecular sentinel (iMS). Machine learning (ML) algorithms have been increasingly adopted for spectral analysis due to their ability to discover underlying patterns and relationships within large and complex data sets. However, the high dimensionality of SERS data poses a challenge for traditional ML techniques, which can be prone to overfitting and poor generalization. Non-negative matrix factorization (NMF) reduces the dimensionality of SERS data while preserving information content. In this paper, we compared the performance of ML methods including convolutional neural network (CNN), support vector regression, and extreme gradient boosting combined with and without NMF for spectral unmixing of four-way multiplexed SERS spectra from iMS assays used for miRNA detection. CNN achieved high accuracy in spectral unmixing. Incorporating NMF before CNN drastically decreased memory and training demands without sacrificing model performance on SERS spectral unmixing. Additionally, models were interpreted using gradient class activation maps and partial dependency plots to understand predictions. These models were used to analyze clinical SERS data from single-plexed iMS in RNA extracted from 17 endoscopic tissue biopsies. CNN and CNN-NMF, trained on multiplexed data, performed most accurately with RMSElabel = 0.101 and 9.68 × 10-2, respectively. We demonstrated that CNN-based ML shows great promise in spectral unmixing of multiplexed SERS spectra, and the effect of dimensionality reduction on performance and training speed.

Performance evaluation of machine learning techniques in predicting cumulative absolute velocity

Cumulative absolute velocity (CAV) is a powerful intensity measure for quantifying potential earthquake damage to structures. Machine learning (ML) methods can provide more accurate and reliable predictions of cumulative absolute velocity due to handling nonlinear relationships, adaptability to changing conditions, automation, efficiency, and the potential for real-time predictions. This study aims to identify machine learning regressions with the highest accuracy for CAV prediction. Several supervised machine learning algorithms were applied and comprehensively compared for performance and accuracy in CAV prediction using the recently compiled Turkish strong-motion database. Support Vector Machine, Linear Regression, Random Forest, Artificial Neural Network, Bayesian Ridge Regression, and Gradient Boosting algorithms were evaluated and compared with traditional Ground Motion Models (GMMs).Two new datasets including 24,667 strong-motion recordings from Turkiye along with global strong-motion recordings are used to build machine learning models. The first dataset contains all recordings of events with 3.5≤MomentMagnitude(Mw)≤7.6, while the second dataset contains only recordings with Mw≥5.5. Moreover, feature selection and outlier detection were performed as preprocessing steps to choose the best of seven CAV estimator parameters in order to boost the ML model performance. To measure the performance of ML methods five evaluation metrics were utilized which are mean square error (MSE), root mean squared error (RMSE), mean absolute error (MAE), mean absolute percent error (MAPE), and correlation coefficient (R).Comparative assessment of the machine learning algorithms suggests that models trained by Mw≥5.5 dataset are quite successful in CAV prediction compared to predictive models trained by the 3.5≤Mw≤7.6 dataset. The result proves that the Gradient Boosting models significantly outperform the other machine learning algorithms in terms of R and RMSE. Machine learning techniques are more successful with user-selected estimators representing the key components that make up earthquake recordings. Finally, the machine learning-based CAV prediction models for Turkiye are compared with available CAV GMMs, and it is observed that the machine learning-based models can predict CAV as successfully as GMMs if there are a sufficient number of recordings for training machine learning algorithms.

Classification of Erythematosquamous Dermatosis by the Method of Random Forest

Machine Learning (ML) methods have found wide applications in dermatology (Chan et al., 2020) [1]. Thomsen, Iversen, Titlestad & Winther (2020) reviewed 2175 publications and found that the most common usage of ML methods was in the binary classification of malignant melanoma from images [2]. Adamson and Smith have a word of advice about usage of ML methods in diagnosis of skin diseases that inclusivity must be kept in mind for classification results to be accurate [3]. Steele et al. searched PubMed, Embase, and CENTRAL, and found that the performance of ML methods was variable, and overall accuracy measure was not a good measure for sub-group accuracy [4].

A machine learning approach for hierarchical classification of software requirements

Context:Classification of software requirements into different categories is a critically important task in requirements engineering (RE). Developing machine learning (ML) approaches for requirements classification has attracted great interest in the RE community since the 2000s. Objective:This paper aims to address two related problems that have been challenging real-world applications of ML approaches: the problems of class imbalance and high dimensionality with low sample size data (HDLSS). These problems can greatly degrade the classification performance of ML methods. Methods:The paper proposes HC4RC, a novel ML approach for multiclass classification of requirements. HC4RC solves the aforementioned problems through semantic-role based feature selection, dataset decomposition and hierarchical classification. We experimentally compare the effectiveness of HC4RC with three closely related approaches — two of which are based on a traditional statistical classification model whereas one using an advanced deep learning model. Results:Our experiment shows: (1) The class imbalance and HDLSS problems present a challenge to both traditional and advanced ML approaches. (2) The HC4RC approach is simple to use and can effectively address the class imbalance and HDLSS problems compared to similar approaches. Conclusion:This paper makes an important practical contribution to addressing the class imbalance and HDLSS problems in multiclass classification of software requirements.

Applications of machine learning in friction stir welding: Prediction of joint properties, real-time control and tool failure diagnosis

Machine learning (ML) methods have received immense attention as potential models for modeling different manufacturing systems. This paper presents a comprehensive review on the applications of ML methods in friction stir welding (FSW) field. Five main topics have been discussed: prediction of the joint properties, integration between ML and finite element methods, real-time control of FSW process, tool failure diagnosis, and incorporation between metaheuristic optimization techniques and ML methods. The common used ML methods such as multi-linear regression, K-nearest neighbor, random forest algorithm, Gaussian process regression, artificial neural network, support vector machine, radial basis function neural network, fuzzy system, adaptive neuro-fuzzy inference system, and random vector functional link are explained. Then, different statistical measures used to evaluate the performance of ML methods are presented. Finally, the applications of ML methods in FSW field are discussed. Important conclusions are drawn and future prospects are suggested.

Identification of species-specific RNA N6-methyladinosine modification sites from RNA sequences.

N6-methyladinosine (m6A) modification is the most abundant co-transcriptional modification in eukaryotic RNA and plays important roles in cellular regulation. Traditional high-throughput sequencing experiments used to explore functional mechanisms are time-consuming and labor-intensive, and most of the proposed methods focused on limited species types. To further understand the relevant biological mechanisms among different species with the same RNA modification, it is necessary to develop a computational scheme that can be applied to different species. To achieve this, we proposed an attention-based deep learning method, adaptive-m6A, which consists of convolutional neural network, bi-directional long short-term memory and an attention mechanism, to identify m6A sites in multiple species. In addition, three conventional machine learning (ML) methods, including support vector machine, random forest and logistic regression classifiers, were considered in this work. In addition to the performance of ML methods for multi-species prediction, the optimal performance of adaptive-m6A yielded an accuracy of 0.9832 and the area under the receiver operating characteristic curve of 0.98. Moreover, the motif analysis and cross-validation among different species were conducted to test the robustness of one model towards multiple species, which helped improve our understanding about the sequence characteristics and biological functions of RNA modifications in different species.

Genomic prediction in plants: opportunities for ensemble machine learning based approaches.

Background: Many studies have demonstrated the utility of machine learning (ML) methods for genomic prediction (GP) of various plant traits, but a clear rationale for choosing ML over conventionally used, often simpler parametric methods, is still lacking. Predictive performance of GP models might depend on a plethora of factors including sample size, number of markers, population structure and genetic architecture. Methods: Here, we investigate which problem and dataset characteristics are related to good performance of ML methods for genomic prediction. We compare the predictive performance of two frequently used ensemble ML methods (Random Forest and Extreme Gradient Boosting) with parametric methods including genomic best linear unbiased prediction (GBLUP), reproducing kernel Hilbert space regression (RKHS), BayesA and BayesB. To explore problem characteristics, we use simulated and real plant traits under different genetic complexity levels determined by the number of Quantitative Trait Loci (QTLs), heritability ( h 2 and h 2 e ), population structure and linkage disequilibrium between causal nucleotides and other SNPs. Results: Decision tree based ensemble ML methods are a better choice for nonlinear phenotypes and are comparable to Bayesian methods for linear phenotypes in the case of large effect Quantitative Trait Nucleotides (QTNs). Furthermore, we find that ML methods are susceptible to confounding due to population structure but less sensitive to low linkage disequilibrium than linear parametric methods. Conclusions: Overall, this provides insights into the role of ML in GP as well as guidelines for practitioners.

Forecasting the Capacity of Open-Ended Pipe Piles Using Machine Learning

Pile design is an essential component of geotechnical engineering practice, and pipe piles, in particular, are increasingly being used for the support of a variety of infrastructure projects. These piles are being used with dimensions that exceed those used in the development of the most widely used design approaches. At the same time, the growth in pile dimensions calls for the evolution of the state-of-the-art at a similar pace. The objective of this study is to provide an improved prediction of pile capacity. A database of 112 load tests on pipe piles ranging in diameter from 10 to 100 in. (0.25–2.5 m) and in length from 10 to 320 ft. (3–98 m) was employed in this study. First, design capacities were computed using four popular design methods and compared to capacities interpreted from a load test. For the employed dataset, the Revised Lambda method was found to best predict capacities of pipe piles obtained from a load test, among the four examined methods, and was thus employed as a reference standard for assessing the performance of ML methods. Next, eight ML regression models were trained to compute the capacity of pipe piles. Several trained ML models predicted capacities for the testing data set on par with the Revised Lambda method, and three were selected for further investigation. A variety of pile dimensions and soil properties were examined as input properties for ML and the trained models performed surprisingly well with only the pile dimensions used as input. In addition, ML models exhibited satisfactory diameter and length effects, which have been areas of concern for some traditional design approaches. The work thus demonstrates the feasibility of employing machine learning (ML) for determining the capacity of pipe piles. A web application was also developed as a tool for forecasting the capacity of pipe piles using ML.

The application of machine learning in nanoparticle treated water: A review

Pollution from industrial effluents and domestic waste are two of the most common sources of environmental pollutants. Due to the rising population and manufacturing industries, large amounts of pollutants were produced daily. Therefore, enhancements in wastewater treatment to render treated wastewater and provide effective solutions are essential to return clean and safe water to be reused in the industrial, agricultural, and domestic sectors. Nanotechnology has been proven as an alternative approach to overcoming the existing water pollution issue. Nanoparticles exhibit high aspect ratios, large pore volumes, electrostatic properties, and high specific surfaces, which explains their efficiency in removing pollutants such as dyes, pesticides, heavy metals, oxygen-demanding wastes, and synthetic organic chemicals. Machine learning (ML) is a powerful tool to conduct the model and prediction of the adverse biological and environmental effects of nanoparticles in wastewater treatment. In this review, the application of ML in nanoparticle-treated water on different pollutants has been studied and it was discovered that the removal of the pollutants could be predicted through the mathematical approach which included ML. Further comparison of ML method can be carried out to assess the prediction performance of ML methods on pollutants removal. Moreover, future studies regarding the nanotoxicity, synthesis process, and reusability of nanoparticles are also necessary to take into consideration to safeguard the environment.

Monitoring mechanical behaviors of CLT connections under reciprocating loading based on PZT-enabled active sensing and machine learning algorithms

Monitoring the mechanical behaviors of cross-laminated timber (CLT) connections is of great importance to the condition assessment of timber structures. To date, numerous research works have demonstrated that Lead Zirconate Titanate (PZT)-enabled active sensing approaches can achieve structural healthy state monitoring under monotonic loads, whereas their effectiveness for reciprocating loads still needs to be further studied. Moreover, traditional PZT-enabled active sensing approaches depend on prior knowledge and human judgment, restricting their field applications. Based on the above background, this research proposes an innovative method to monitor the mechanical behaviors of CLT connections under reciprocating loading by integrating PZT-enabled active sensing and eight machine learning (ML) approaches. Meanwhile, a new damage index based on wavelet packet decomposition and multiple signal path fusion is designed to improve the performance of ML methods. Finally, cyclic loading tests on CLT connections are conducted to demonstrate the outstanding capabilities of the proposed method than conventional PZT-enabled active sensing approaches.

Application of convex hull analysis for the evaluation of data heterogeneity between patient populations of different origin and implications of hospital bias in downstream machine-learning-based data processing: A comparison of 4 critical-care patient datasets.

Machine learning (ML) models are developed on a learning dataset covering only a small part of the data of interest. If model predictions are accurate for the learning dataset but fail for unseen data then generalization error is considered high. This problem manifests itself within all major sub-fields of ML but is especially relevant in medical applications. Clinical data structures, patient cohorts, and clinical protocols may be highly biased among hospitals such that sampling of representative learning datasets to learn ML models remains a challenge. As ML models exhibit poor predictive performance over data ranges sparsely or not covered by the learning dataset, in this study, we propose a novel method to assess their generalization capability among different hospitals based on the convex hull (CH) overlap between multivariate datasets. To reduce dimensionality effects, we used a two-step approach. First, CH analysis was applied to find mean CH coverage between each of the two datasets, resulting in an upper bound of the prediction range. Second, 4 types of ML models were trained to classify the origin of a dataset (i.e., from which hospital) and to estimate differences in datasets with respect to underlying distributions. To demonstrate the applicability of our method, we used 4 critical-care patient datasets from different hospitals in Germany and USA. We estimated the similarity of these populations and investigated whether ML models developed on one dataset can be reliably applied to another one. We show that the strongest drop in performance was associated with the poor intersection of convex hulls in the corresponding hospitals' datasets and with a high performance of ML methods for dataset discrimination. Hence, we suggest the application of our pipeline as a first tool to assess the transferability of trained models. We emphasize that datasets from different hospitals represent heterogeneous data sources, and the transfer from one database to another should be performed with utmost care to avoid implications during real-world applications of the developed models. Further research is needed to develop methods for the adaptation of ML models to new hospitals. In addition, more work should be aimed at the creation of gold-standard datasets that are large and diverse with data from varied application sites.

A review of machine learning techniques for identifying weeds in corn

Weeds pose a major challenge in achieving high yield production in corn. The use of herbicides although effective can be expensive and their excessive use poses ecological concerns and herbicide resistance. Precise identification of weeds using Machine Learning (ML) models significantly reduces the use of herbicides. In this study, we provide a brief overview of the important ML methods used for identifying weeds in corn i.e., classification and object detection. The various metrics that are used for the evaluation of the performance of ML methods are also discussed. In the end, we identify some important research gaps which warrant future investigation. Most ML methods for the identification of weeds use digital images as input data, however, in some cases, hyperspectral data were used. Most of the current studies employ support vector machines and neural networks for the identification of weeds. Classification accuracy and F1 score are the two most frequently used accuracy metrics to evaluate the performance of ML models used. Future research on the identification of weeds may focus on improving the data volume using data augmentation, transfer learning to benefit from existing models, and interpretability of neural networks to avoid overfitting and make models more transparent.

Genomic prediction in plants: opportunities for ensemble machine learning based approaches.

Background: Many studies have demonstrated the utility of machine learning (ML) methods for genomic prediction (GP) of various plant traits, but a clear rationale for choosing ML over conventionally used, often simpler parametric methods, is still lacking. Predictive performance of GP models might depend on a plethora of factors including sample size, number of markers, population structure and genetic architecture. Methods: Here, we investigate which problem and dataset characteristics are related to good performance of ML methods for genomic prediction. We compare the predictive performance of two frequently used ensemble ML methods (Random Forest and Extreme Gradient Boosting) with parametric methods including genomic best linear unbiased prediction (GBLUP), reproducing kernel Hilbert space regression (RKHS), BayesA and BayesB. To explore problem characteristics, we use simulated and real plant traits under different genetic complexity levels determined by the number of Quantitative Trait Loci (QTLs), heritability ( h 2 and h 2 e ), population structure and linkage disequilibrium between causal nucleotides and other SNPs. Results: Decision tree based ensemble ML methods are a better choice for nonlinear phenotypes and are comparable to Bayesian methods for linear phenotypes in the case of large effect Quantitative Trait Nucleotides (QTNs). Furthermore, we find that ML methods are susceptible to confounding due to population structure but less sensitive to low linkage disequilibrium than linear parametric methods. Conclusions: Overall, this provides insights into the role of ML in GP as well as guidelines for practitioners.

Performance Analysis of Machine Learning Methods with Class Imbalance Problem in Android Malware Detection

Due to the exponential rise of mobile technology, a slew of new mobile security concerns has surfaced recently. To address the hazards connected with malware, many approaches have been developed. Signature-based detection is the most widely used approach for detecting Android malware. This approach has the disadvantage of being unable to identify unknown malware. As a result of this issue, machine learning (ML) for identifying and categorising malware apps was created. Conventional ML methods are concerned with increasing classification accuracy. However, the standard classification method performs poorly in recognising malware applications due to the unbalanced real-world datasets. In this study, an empirical analysis of the detection performance of ML methods in the presence of class imbalance is conducted. Specifically, eleven (11) ML methods with diverse computational complexities were investigated. Also, a synthetic minority oversampling technique (SMOTE) and random undersampling (RUS) are deployed to address the class imbalance in the Android malware datasets. The experimented ML methods are tested using the Malgenome and Drebin Android malware datasets that contain features gathered from both static and dynamic malware approaches. According to the experimental findings, the performance of each experimented ML method varies across the datasets. Moreover, the presence of class imbalance deteriorated the performance of the ML methods as their performances were amplified with the deployment of data sampling methods (SMOTE and RUS) used to alleviate the class imbalance problem. Besides, ML models with SMOTE technique are superior to other experimented methods. It is therefore recommended to address the inherent class imbalance problem in Android Malware detection.

On the interpretability of machine learning methods in crash frequency modeling and crash modification factor development

Machine learning (ML) model interpretability has attracted much attention recently given the promising performance of ML methods in crash frequency studies. Extracting accurate relationship between risk factors and crash frequency is important for understanding the causal effects of risk factors and developing safety countermeasures. However, there is no study that comprehensively summarizes ML model interpretation methods and provides guidance for safety researchers and practitioners. This research aims to fill this gap. Model-based and post-hoc ML interpretation methods are critically evaluated and compared to study their suitability in crash frequency modeling. These methods include classification and regression tree (CART), multivariate adaptive regression splines (MARS), Local Interpretable Model-agnostic Explanations (LIME), Local Sensitivity Analysis (LSA), Partial Dependence Plots (PDP), Global Sensitivity Analysis (GSA), and SHapley Additive exPlanations (SHAP). Model-based interpretation methods cannot reveal the detailed interaction relationships among risk factors. LIME can only be used to analyze the effects of a risk factor at the prediction level. LSA and PDP assume that different risk factors are independently distributed. Both GSA and SHAP can account for the potential correlation among risk factors. However, only SHAP can visualize the detailed relationships between crash outcomes and risk factors. This study also demonstrates the potential and benefits of using ML and SHAP to derive Crash Modification Factors (CMF). Finally, it is emphasized that statistical and ML models may not directly differentiate causation from correlation. Understanding the differences between them is critical for developing reliable safety countermeasures.